{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ff5e92e4",
   "metadata": {
    "id": "9-a3wNZ9mxWT"
   },
   "source": [
    "# Multivariate Probabilistic Time Series Forecasting with Informer\n",
    "\n",
    "## Introduction\n",
    "\n",
    "A few months ago we introduced the [Time Series Transformer](https://huggingface.co/blog/time-series-transformers), which is the vanilla Transformer ([Vaswani et al., 2017](https://arxiv.org/abs/1706.03762)) applied to forecasting, and showed an example for the **univariate** probabilistic forecasting task (i.e. predicting each time series' 1-d distribution individually). In this post we introduce the _Informer_ model ([Zhou, Haoyi, et al., 2021](https://arxiv.org/abs/2012.07436)), AAAI21 best paper which is [now available](https://huggingface.co/docs/transformers/main/en/model_doc/informer) in 🤗 Transformers. We will show how to use the Informer model for the **multivariate** probabilistic forecasting task, i.e., predicting the distribution of a future **vector** of time-series target values. Note that this will also work for the vanilla Time Series Transformer model.\n",
    "\n",
    "##  Multivariate Probabilistic Time Series Forecasting\n",
    "\n",
    "As far as the modeling aspect of probabilistic forecasting is concerned, the Transformer/Informer will require no change when dealing with multivariate time series. In both the univariate and multivariate setting, the model will receive a sequence of vectors and thus the only change is on the output or emission side.\n",
    "\n",
    "Modeling the full joint conditional distribution of high dimensional data can get computationally expensive and thus methods resort to some approximation of the distribution, the easiest being to model the data as an independent distribution from the same family, or some low-rank approximation to the full covariance, etc. Here we will just resort to the independent (or diagonal) emissions which are supported for the families of distributions we have implemented [here](https://huggingface.co/docs/transformers/main/en/internal/time_series_utils).\n",
    "\n",
    "## Informer - Under The Hood\n",
    "\n",
    "Based on the vanilla Transformer ([Vaswani et al., 2017](https://arxiv.org/abs/1706.03762)), Informer employs two major improvements. To understand these improvements, let's recall the drawbacks of the vanilla Transformer:\n",
    "\n",
    "1. **Quadratic computation of canonical self-attention:** The vanilla Transformer has a computational complexity of $O(T^2 D)$ where $T$ is the time series length and $D$ is the dimension of the hidden states. For long sequence time-series forecasting (also known as the _LSTF problem_), this might be really computationally expensive. To solve this problem, Informer employs a new self-attention mechanism called _ProbSparse_ attention, which has $O(T \\log T)$ time and space complexity.\n",
    "1. **Memory bottleneck when stacking layers:** When stacking $N$ encoder/decoder layers, the vanilla Transformer has a memory usage of $O(N T^2)$, which limits the model's capacity for long sequences. Informer uses a _Distilling_ operation, for reducing the input size between layers into its half slice. By doing so, it reduces the whole memory usage to be $O(N\\cdot T \\log T)$.\n",
    "\n",
    "As you can see, the motivation for the Informer model is similar to Longformer ([Beltagy et el., 2020](https://arxiv.org/abs/2004.05150)), Sparse Transformer ([Child et al., 2019](https://arxiv.org/abs/1904.10509)) and other NLP papers for reducing the quadratic complexity of the self-attention mechanism **when the input sequence is long**. Now, let's dive into _ProbSparse_ attention and the _Distilling_ operation with code examples. \n",
    "\n",
    "### ProbSparse Attention\n",
    "\n",
    "The main idea of ProbSparse is that the canonical self-attention scores form a long-tail distribution, where the \"active\" queries lie in the \"head\" scores and \"lazy\" queries lie in the \"tail\" area. By \"active\" query we mean a query $q_i$ such that the dot-product $\\langle q_i,k_i \\rangle$ **contributes** to the major attention, whereas a \"lazy\" query forms a dot-product which generates  **trivial** attention. Here, $q_i$ and $k_i$ are the $i$-th rows in $Q$ and $K$ attention matrices respectively. \n",
    "\n",
    "| ![informer_full_vs_sparse_attention](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/informer/informer_full_vs_sparse_attention.png) |\n",
    "|:--:|\n",
    "|  Vanilla self attention vs ProbSparse attention from [Autoformer (Wu, Haixu, et al., 2021)](https://wuhaixu2016.github.io/pdf/NeurIPS2021_Autoformer.pdf) |\n",
    "\n",
    "Given the idea of \"active\" and \"lazy\" queries, the ProbSparse attention selects the \"active\" queries, and creates a reduced query matrix $Q_{reduced}$ which is used to calculate the attention weights in $O(T \\log T)$. Let's see this more in detail with a code example. \n",
    "    \n",
    "Recall the canonical self-attention formula:\n",
    "\n",
    "$$\n",
    "\\textrm{Attention}(Q, K, V) = \\textrm{softmax}(\\frac{QK^T}{\\sqrt{d_k}} )V\n",
    "$$\n",
    "\n",
    "Where $Q\\in \\mathbb{R}^{L_Q \\times d}, K\\in \\mathbb{R}^{L_K \\times d}, V\\in \\mathbb{R}^{L_V \\times d}$. Note that in practice, the input length of queries and keys are typically equivalent in the self-attention computation, i.e. $L_Q = L_K = T$ where $T$ is the time series length. Therefore, the $QK^T$ multiplication takes $O(T^2 \\cdot d)$ computational complexity. In ProbSparse attention, our goal is to create a new $Q_{reduce}$ matrix and define:\n",
    "\n",
    "$$\n",
    "\\textrm{ProbSparseAttention}(Q, K, V) = \\textrm{softmax}(\\frac{Q_{reduce}K^T}{\\sqrt{d_k}} )V\n",
    "$$\n",
    "\n",
    "where the $Q_{reduce}$ matrix only selects the Top $u$ \"active\" queries. Here, $u = c \\cdot \\log L_Q$ and $c$ called the _sampling factor_ hyperparameter for the ProbSparse attention. Since $Q_{reduce}$ selects only the Top $u$ queries, its size is $c\\cdot \\log L_Q \\times d$, so the multiplication $Q_{reduce}K^T$ takes only $O(L_K \\log L_Q) = O(T \\log T)$.\n",
    "\n",
    "This is good! But how can we select the $u$ \"active\" queries to create $Q_{reduce}$? Let's define the _Query Sparsity Measurement_.\n",
    "\n",
    "#### Query Sparsity Measurement\n",
    "Query Sparsity Measurement $M(q_i, K)$ is used for selecting the $u$ \"active\" queries $q_i$ in $Q$ to create $Q_{reduce}$. In theory, the dominant $\\langle q_i,k_i \\rangle$ pairs encourage the \"active\" $q_i$'s probability distribution **away** from the uniform distribution as can be seen in the figure below. Hence, the [KL divergence](https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence) between the actual queries distribution and the uniform distribution is used to define the sparsity measurement. \n",
    "\n",
    "| ![informer_probsparse](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/informer/informer_probsparse.png) | \n",
    "|:--:|\n",
    "| The illustration of ProbSparse Attention from official [repository](https://github.com/zhouhaoyi/Informer2020)|\n",
    "\n",
    "\n",
    "In practice, the measurement is defined as:\n",
    "\n",
    "$$\n",
    "M(q_i, K) = \\max_j \\frac{q_ik_j^T}{\\sqrt{d}}-\\frac{1}{L_k} \\sum_{j=1}^{L_k}\\frac{q_ik_j^T}{\\sqrt{d}}\n",
    "$$\n",
    "\n",
    "\n",
    "The important thing to understand here is when $M(q_i, K)$ is larger, the query $q_i$ should be in $Q_{reduce}$ and vice versa.\n",
    "\n",
    "But how can we calculate the term $q_ik_j^T$ in non-quadratic time? Recall that most of the dot-product $\\langle q_i,k_i \\rangle$ generate either way the trivial attention (i.e. long-tail distribution property), so it is enough to randomly sample a subset of keys from $K$, which will be called `K_sample` in the code.\n",
    "\n",
    "Now, we are ready to see the code of `probsparse_attention`:\n",
    "    \n",
    "```python\n",
    "from torch import nn\n",
    "import math\n",
    "\n",
    "\n",
    "def probsparse_attention(query_states, key_states, value_states, sampling_factor=5):\n",
    "    \"\"\"\n",
    "    Compute the probsparse self-attention.\n",
    "    Input shape: Batch x Time x Channel\n",
    "\n",
    "    Note the additional `sampling_factor` input.\n",
    "    \"\"\"\n",
    "    # get input sizes with logs\n",
    "    L_K = key_states.size(1)\n",
    "    L_Q = query_states.size(1)\n",
    "    log_L_K = np.ceil(np.log1p(L_K)).astype(\"int\").item()\n",
    "    log_L_Q = np.ceil(np.log1p(L_Q)).astype(\"int\").item()\n",
    "\n",
    "    # calculate a subset of samples to slice from K and create Q_K_sample\n",
    "    U_part = min(sampling_factor * L_Q * log_L_K, L_K)\n",
    "\n",
    "    # create Q_K_sample (the q_i * k_j^T term in the sparsity measurement)\n",
    "    index_sample = torch.randint(0, L_K, (U_part,))\n",
    "    K_sample = key_states[:, index_sample, :]\n",
    "    Q_K_sample = torch.bmm(query_states, K_sample.transpose(1, 2))\n",
    "\n",
    "    # calculate the query sparsity measurement with Q_K_sample\n",
    "    M = Q_K_sample.max(dim=-1)[0] - torch.div(Q_K_sample.sum(dim=-1), L_K)\n",
    "\n",
    "    # calculate u to find the Top-u queries under the sparsity measurement\n",
    "    u = min(sampling_factor * log_L_Q, L_Q)\n",
    "    M_top = M.topk(u, sorted=False)[1]\n",
    "\n",
    "    # calculate Q_reduce as query_states[:, M_top]\n",
    "    dim_for_slice = torch.arange(query_states.size(0)).unsqueeze(-1)\n",
    "    Q_reduce = query_states[dim_for_slice, M_top]  # size: c*log_L_Q x channel\n",
    "\n",
    "    # and now, same as the canonical\n",
    "    d_k = query_states.size(-1)\n",
    "    attn_scores = torch.bmm(Q_reduce, key_states.transpose(-2, -1))  # Q_reduce x K^T\n",
    "    attn_scores = attn_scores / math.sqrt(d_k)\n",
    "    attn_probs = nn.functional.softmax(attn_scores, dim=-1)\n",
    "    attn_output = torch.bmm(attn_probs, value_states)\n",
    "\n",
    "    return attn_output, attn_scores\n",
    "```\n",
    "Note that in the implementation, $U_{part}$ contain $L_Q$ in the calculation for stability issues (see [this disccusion](https://discuss.huggingface.co/t/probsparse-attention-in-informer/34428) for more information).\n",
    "\n",
    "We did it! Please be aware that this is only a partial implementation of the `probsparse_attention`, and the full implementation can be found in 🤗 Transformers.\n",
    "\n",
    "### Distilling\n",
    "\n",
    "Because of the ProbSparse self-attention, the encoder’s feature map has some redundancy that can be removed. Therefore,\n",
    "the distilling operation is used to reduce the input size between encoder layers into its half slice, thus in theory removing this redundancy. In practice, Informer's \"distilling\" operation just adds 1D convolution layers with max pooling between each of the encoder layers. Let $X_n$ be the output of the $n$-th encoder layer, the distilling operation is then defined as:\n",
    "\n",
    "\n",
    "$$\n",
    "X_{n+1} = \\textrm{MaxPool} ( \\textrm{ELU}(\\textrm{Conv1d}(X_n))\n",
    "$$\n",
    "\n",
    "\n",
    "Let's see this in code:\n",
    "    \n",
    "```python\n",
    "from torch import nn\n",
    "\n",
    "# ConvLayer is a class with forward pass applying ELU and MaxPool1d\n",
    "def informer_encoder_forward(x_input, num_encoder_layers=3, distil=True):\n",
    "    # Initialize the convolution layers\n",
    "    if distil:\n",
    "        conv_layers = nn.ModuleList([ConvLayer() for _ in range(num_encoder_layers - 1)])\n",
    "        conv_layers.append(None)\n",
    "    else:\n",
    "        conv_layers = [None] * num_encoder_layers\n",
    "    \n",
    "    # Apply conv_layer between each encoder_layer\n",
    "    for encoder_layer, conv_layer in zip(encoder_layers, conv_layers):\n",
    "        output = encoder_layer(x_input)\n",
    "        if conv_layer is not None:\n",
    "            output = conv_layer(loutput)\n",
    "    \n",
    "    return output\n",
    "```\n",
    "    \n",
    "By reducing the input of each layer by two, we get a memory usage of $O(N\\cdot T \\log T)$ instead of $O(N\\cdot T^2)$ where $N$ is the number of encoder/decoder layers. This is what we wanted!\n",
    "    \n",
    "The Informer model in [now available](https://huggingface.co/docs/transformers/main/en/model_doc/informer) in the 🤗 Transformers library, and simply called `InformerModel`. In the sections below, we will show how to train this model on a custom multivariate time-series dataset.\n",
    "\n",
    "\n",
    "## Set-up Environment\n",
    "\n",
    "First, let's install the necessary libraries: 🤗 Transformers, 🤗 Datasets, 🤗 Evaluate,  🤗 Accelerate and [GluonTS](https://github.com/awslabs/gluonts).\n",
    "\n",
    "As we will show, GluonTS will be used for transforming the data to create features as well as for creating appropriate training, validation and test batches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7AkJMJAunLP9",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7AkJMJAunLP9",
    "outputId": "b57a63c6-d3da-4034-b081-e3f2da24a8f4"
   },
   "outputs": [],
   "source": [
    "%pip install -q transformers datasets evaluate accelerate gluonts ujson"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4m_7_MKqmzfE",
   "metadata": {
    "id": "4m_7_MKqmzfE"
   },
   "source": [
    "## Load Dataset\n",
    "\n",
    "In this blog post, we'll use the `traffic_hourly` dataset, which is available on the [Hugging Face Hub](https://huggingface.co/datasets/monash_tsf). This dataset contains the San Francisco Traffic dataset used by [Lai et al. (2017)](https://arxiv.org/abs/1703.07015). It contains 862 hourly time series showing the road occupancy rates in the range $[0, 1]$ on the San Francisco Bay area freeways from 2015 to 2016.\n",
    "\n",
    "This dataset is part of the [Monash Time Series Forecasting](https://forecastingdata.org/) repository, a collection of time series datasets from a number of domains. It can be viewed as the [GLUE benchmark](https://gluebenchmark.com/) of time series forecasting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d93a3f41",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 264,
     "referenced_widgets": [
      "956dc45f3dde496d94f2d2520f62a241",
      "dc1ad0316a634e0e81eecdd09d271136",
      "5c531b5b3ce4406abde898297f4bc80b",
      "8b30e8b778fb456bbf8af4d6931f415c",
      "d1f7dc29912b40e7ac4a9081a4068836",
      "f8af035c2033430ca41cb3c81719ddc8",
      "682be41e57c341189bdc3e7a6b7246d3",
      "b7e131bcb6454ccbb5de0813a734586a",
      "3e1c8ac5bfac429993d5404f7abf748d",
      "630d2a4aba53498b8e6f40d713fe50d1",
      "313d214425c541b6845f70638f8a5937",
      "2dc9cef88370404184dc1f629d062a04",
      "ed58c5b583f344cdbfea76562259190e",
      "c56009b340984f2296ad2959e8d72f25",
      "b2eeaecdd52343a4977954915f38bc9b",
      "c55cb9a98afe49d59838a3e2d88cd774",
      "672ee2ad96ac42fa90645a4049c7dd16",
      "96acfb4ec404421b8523e39b6a4bc41a",
      "c6fa8ced2f844e28a6e5cf4b31e362b1",
      "b65097925371428eb2b58eec672f9b81",
      "726309e90e0247628487e1f4816e6fd9",
      "e791aeda253f40d3bf28da3aaf0ec3f3",
      "87d63881c376462b88a7703f62495a83",
      "af5add21a56b4ef4945a90fed06d7c89",
      "c0822d9ca57b4b23ba552b93f139d96e",
      "2ec6b12a621f4ce681554397392bdda8",
      "8de92b21ea1b4dfc966d653e0ca7043b",
      "bcaa24789c664ed181eb3af6c93a618e",
      "e4015a34ae9744ceaa57d8b50a024d9a",
      "23207dda3c7c45ef942fb26b6d3e8f4a",
      "0a4bcc907dbc48f3af39fac4c0652bd4",
      "6a39d27a5d5e484385f3d8163665641f",
      "649640a4dacf4edab3aba273d7e09bbf",
      "0d037e1f14794a0fb68ab472b42ee60a",
      "eaa70494878f4eba927868ecbbd034b4",
      "b9f43527572c41849f66e100e4d2df83",
      "f0019a4884124eebb44571ca0a7b262e",
      "a79295d7dc77434282636a0ef665a70b",
      "b0b4e07185574c5eba5144a28c611971",
      "01a6160718434f7bb73bb4a094ee8b0f",
      "a70e5dadc1d547cc90da492ef9b72172",
      "3ff0a924499f414ca7fd03a902b51102",
      "0452728577c4433d9d15eca97045d1f4",
      "53938941490b4afea31c63aad4bdce0a",
      "1076c8e3f8e1413fad4aa5cb16c71c31",
      "dbb568c5b348421a9e2f9f4e07cd5f01",
      "966ca413bd824a77b50a20784e8bd1b4",
      "e81a3f25949247eeb696ccd3b0df3da0",
      "228e4cf452a04b1f8f324be138857c30",
      "addaf03f64e9421ab8debbba4f096d7e",
      "94aa0a38801e4fa0b79150fbd7bcc6cc",
      "044030ba2c5948d2ac6545aea773ef4a",
      "a54e00389ce148829b4db5b7439ae879",
      "435416a295674774a15aa700ff7ce007",
      "d7be4847987e4d6996c56eb42feb78b9",
      "43daab71788e4a5c80687a9e73bf3b67",
      "67f8fc82c7044090a0408294bc1c2d15",
      "7cd0096ebb94429dab18e3226fc4cdbf",
      "0bf375322d12400ab773e85c08cd3e62",
      "e3641728ace74d8ab4742007d1725ee8",
      "3c850c33752443afa6658a1cb997549e",
      "3366b16d73d949508f24d647558db059",
      "741afc0fb9a7469cac4a1d6b0dc130d8",
      "df02d3127e634862b1029787be8908b8",
      "22c8c8ecb39c491798602abe0144eba1",
      "36a6c51643ba4375b314e9395c0aea07",
      "83c39de970e84954b6fa043390b85d4c",
      "009a21c6a2dd4c5da79a7fb3eec8ff43",
      "9059a60d370c4825a0eb1b31c9f9cd0a",
      "028908857f134943af7b47a2503ce403",
      "445f2ef3f71a4573ae453144dd4c5e81",
      "a8b516bf196946a09a55732d937dfe2a",
      "53a3ad2f991648439e57362c4a7a5f60",
      "db1af11912fa4774b718242b8861a18c",
      "2e0511f2f1844758a6f6d10af7233a72",
      "50e91b016b0a4ce69325e8196a09dce6",
      "9c573dc9c14248deb512162af5f9b0fa",
      "ed32d5154c8e45ff91608c8dca664d8e",
      "d2e3caaeded749fc807de397f79e98e7",
      "f9d799cf26574229ad1d77373cb511c5",
      "17a314c74019462e9ba2e923b9235bed",
      "2435607c37f2439aa1f0d545009b375e",
      "09cc068788e64c1fbc1edd0992f43f42",
      "46e05d2996a84b5e8f456adbb6453350",
      "7cc6a3332eaf4e9bb012097ca03c6ef6",
      "7bb9e900006840a7a892a1a48d676560",
      "6b97acade12a49d683ec81855e1bf58a",
      "45b1dbee32a440328f91bf0c398d4a88",
      "048591e682544b57abbcba613c81f9b4",
      "d8a6361f19894be1b730137b0801e4e8",
      "2c375a08b1cb4f71a0a0a169c999aa94",
      "ea1af7f963e848d19ea363331027109b",
      "b68b7b7b072e457980e834113ff0a670",
      "764abccaae30410ba68405f44875c8d6",
      "89dd9177f7cd4edc87e43d5e23c5f587",
      "a3384254326f49eb907bfa6d2032b1a8",
      "68e542f3c052435f9254e135e5c481e8",
      "d0cecf202f814209abe965e2ec13c49e",
      "331752b5913245e380752532ab15f128"
     ]
    },
    "id": "d93a3f41",
    "outputId": "5708bef0-3c78-4f58-9a1f-7db01177cfe2"
   },
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "\n",
    "dataset = load_dataset(\"monash_tsf\", \"traffic_hourly\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sG1ZNNzwpwmg",
   "metadata": {
    "id": "sG1ZNNzwpwmg"
   },
   "source": [
    "As can be seen, the dataset contains 3 splits: train, validation and test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d45ae147",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "d45ae147",
    "outputId": "c8b9ea06-c8ea-443c-8bf2-72acb311be39"
   },
   "outputs": [],
   "source": [
    "dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26KIJLY2qFXI",
   "metadata": {
    "id": "26KIJLY2qFXI"
   },
   "source": [
    "Each example contains a few keys, of which `start` and `target` are the most important ones. Let us have a look at the first time series in the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "oHlRCUPkoN1N",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "oHlRCUPkoN1N",
    "outputId": "fe314df1-b6f8-45b9-b3a9-fe7caf44d30b"
   },
   "outputs": [],
   "source": [
    "train_example = dataset[\"train\"][0]\n",
    "train_example.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "gbVF5vAcqzJG",
   "metadata": {
    "id": "gbVF5vAcqzJG"
   },
   "source": [
    "The `start` simply indicates the start of the time series (as a datetime), and the `target` contains the actual values of the time series.\n",
    "\n",
    "The `start` will be useful to add time related features to the time series values, as extra input to the model (such as \"month of year\"). Since we know the frequency of the data is `hourly`, we know for instance that the second value has the timestamp `2015-01-01 01:00:01`, `2015-01-01 02:00:01`, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1PDt8bvwoUbN",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1PDt8bvwoUbN",
    "outputId": "774b6c03-b1c9-4e7a-f3b8-a31a23abdb51"
   },
   "outputs": [],
   "source": [
    "print(train_example[\"start\"])\n",
    "print(len(train_example[\"target\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "DfkPxhCkquKL",
   "metadata": {
    "id": "DfkPxhCkquKL"
   },
   "source": [
    "The validation set contains the same data as the training set, just for a `prediction_length` longer amount of time. This allows us to validate the model's predictions against the ground truth.\n",
    "\n",
    "The test set is again one `prediction_length` longer data compared to the validation set (or some multiple of  `prediction_length` longer data compared to the training set for testing on multiple rolling windows)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eRQhm4EGpa0y",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eRQhm4EGpa0y",
    "outputId": "bb5e8298-cd62-4cd8-bb54-62377d8f8ca1"
   },
   "outputs": [],
   "source": [
    "validation_example = dataset[\"validation\"][0]\n",
    "validation_example.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "x5PI_Jn7rDfj",
   "metadata": {
    "id": "x5PI_Jn7rDfj"
   },
   "source": [
    "The initial values are exactly the same as the corresponding training example. However, this example has `prediction_length=48` (48 hours, or 2 days) additional values compared to the training example. Let us verify it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "__j4Z5Ohp8gg",
   "metadata": {
    "id": "__j4Z5Ohp8gg"
   },
   "outputs": [],
   "source": [
    "freq = \"1H\"\n",
    "prediction_length = 48\n",
    "\n",
    "assert len(train_example[\"target\"]) + prediction_length == len(\n",
    "    dataset[\"validation\"][0][\"target\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "PGq2e9D-rhtg",
   "metadata": {
    "id": "PGq2e9D-rhtg"
   },
   "source": [
    "Let's visualize this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cYDyml0tsnlL",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 265
    },
    "id": "cYDyml0tsnlL",
    "outputId": "0942f1ea-b333-4af4-d0d2-8cd3034b104d"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "num_of_samples = 150\n",
    "\n",
    "figure, axes = plt.subplots()\n",
    "axes.plot(train_example[\"target\"][-num_of_samples:], color=\"blue\")\n",
    "axes.plot(\n",
    "    validation_example[\"target\"][-num_of_samples - prediction_length :],\n",
    "    color=\"red\",\n",
    "    alpha=0.5,\n",
    ")\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4pGV6_ZduUaA",
   "metadata": {
    "id": "4pGV6_ZduUaA"
   },
   "source": [
    "Let's split up the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7eb15a6a",
   "metadata": {
    "id": "7eb15a6a"
   },
   "outputs": [],
   "source": [
    "train_dataset = dataset[\"train\"]\n",
    "test_dataset = dataset[\"test\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0125722c",
   "metadata": {
    "id": "0125722c"
   },
   "source": [
    "## Update `start` to `pd.Period`\n",
    "\n",
    "The first thing we'll do is convert the `start` feature of each time series to a pandas `Period` index using the data's `freq`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "249a9da4",
   "metadata": {
    "id": "249a9da4"
   },
   "outputs": [],
   "source": [
    "from functools import lru_cache\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "@lru_cache(10_000)\n",
    "def convert_to_pandas_period(date, freq):\n",
    "    return pd.Period(date, freq)\n",
    "\n",
    "\n",
    "def transform_start_field(batch, freq):\n",
    "    batch[\"start\"] = [convert_to_pandas_period(date, freq) for date in batch[\"start\"]]\n",
    "    return batch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "D7goNkUB1MPB",
   "metadata": {
    "id": "D7goNkUB1MPB"
   },
   "source": [
    "We now use `datasets`' [`set_transform`](https://huggingface.co/docs/datasets/v2.7.0/en/package_reference/main_classes#datasets.Dataset.set_transform) functionality to do this on-the-fly in place:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b43c7551",
   "metadata": {
    "id": "b43c7551"
   },
   "outputs": [],
   "source": [
    "from functools import partial\n",
    "\n",
    "train_dataset.set_transform(partial(transform_start_field, freq=freq))\n",
    "test_dataset.set_transform(partial(transform_start_field, freq=freq))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "909db77b",
   "metadata": {},
   "source": [
    "Now, let's convert the dataset into a multivariate time series using the `MultivariateGrouper` from GluonTS. This grouper will convert the individual 1-dimensional time series into a single 2D matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0088e817",
   "metadata": {},
   "outputs": [],
   "source": [
    "from gluonts.dataset.multivariate_grouper import MultivariateGrouper\n",
    "\n",
    "num_of_variates = len(train_dataset)\n",
    "\n",
    "train_grouper = MultivariateGrouper(max_target_dim=num_of_variates)\n",
    "test_grouper = MultivariateGrouper(\n",
    "    max_target_dim=num_of_variates,\n",
    "    num_test_dates=len(test_dataset)\n",
    "    // num_of_variates,  # number of rolling test windows\n",
    ")\n",
    "\n",
    "multi_variate_train_dataset = train_grouper(train_dataset)\n",
    "multi_variate_test_dataset = test_grouper(test_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "469f5ec3",
   "metadata": {},
   "source": [
    "Note that now the target is 2 dimensional, where the first dim is the number of variates (number of time-series) and the second is the time-series values (time dimension): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12a8e9c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "multi_variate_train_example = multi_variate_train_dataset[0]\n",
    "print(\n",
    "    f\"multi_variate_train_example['target'].shape = {multi_variate_train_example['target'].shape}\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50efb119",
   "metadata": {
    "id": "50efb119"
   },
   "source": [
    "## Define the model\n",
    "\n",
    "Next, let's instantiate a model. The model will be trained from scratch, hence we won't use the `from_pretrained` method here, but rather randomly initialize the model from a [`config`](https://huggingface.co/docs/transformers/main/en/model_doc/informer#transformers.InformerConfig).\n",
    "\n",
    "We specify a couple of additional parameters to the model:\n",
    "- `prediction_length` (in our case, `48` hours): this is the horizon that the decoder of the Informer will learn to predict for;\n",
    "- `context_length`: the model will set the `context_length` (input of the encoder) equal to the `prediction_length`, if no `context_length` is specified;\n",
    "- `lags` for a given frequency: these specify an efficient \"look back\" mechanism, where we concatenate values from the past to the current values as additional features, e.g. for a `Daily` frequency we might consider a look back of `[1, 7, 30, ...]` or for `Minute` data we might consider `[1, 30, 60, 60*24, ...]` etc.;\n",
    "- the number of time features: in our case, this will be `5` as we'll add `HourOfDay`, `DayOfWeek`, ..., and `Age` features (see below).\n",
    "\n",
    "Let us check the default lags provided by GluonTS for the given frequency (\"hourly\"):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6CF4M8Ms7W-q",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "6CF4M8Ms7W-q",
    "outputId": "4481a9bd-343c-45bd-b569-88d233ab8d40"
   },
   "outputs": [],
   "source": [
    "from gluonts.time_feature import get_lags_for_frequency\n",
    "\n",
    "lags_sequence = get_lags_for_frequency(freq)\n",
    "print(lags_sequence)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "q6ZuFx8yCSAM",
   "metadata": {
    "id": "q6ZuFx8yCSAM"
   },
   "source": [
    "This means that this would look back up to 721 hours (~30 days) for each time step, as additional features. However, the resulting feature vector would end up being of size `len(lags_sequence)*num_of_variates` which for our case will be 34480! This is not going to work so we will use our own sensible lags.\n",
    "\n",
    "Let us also check the default time features which GluonTS provides us:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "VlP_0E5I76lg",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VlP_0E5I76lg",
    "outputId": "03d14e28-8609-4c8e-ef7c-efb17f41076a"
   },
   "outputs": [],
   "source": [
    "from gluonts.time_feature import time_features_from_frequency_str\n",
    "\n",
    "time_features = time_features_from_frequency_str(freq)\n",
    "print(time_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "m0_f7lm9CbNG",
   "metadata": {
    "id": "m0_f7lm9CbNG"
   },
   "source": [
    "In this case, there are four additional features, namely \"hour of day\", \"day of week\", \"day of month\" and \"day of year\". This means that for each time step, we'll add these features as a scalar values. For example, consider the timestamp `2015-01-01 01:00:01`. The four additional features will be:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99d3d336",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pandas.core.arrays.period import period_array\n",
    "\n",
    "timestamp = pd.Period(\"2015-01-01 01:00:01\", freq=freq)\n",
    "timestamp_as_index = pd.PeriodIndex(data=period_array([timestamp]))\n",
    "additional_features = [\n",
    "    (time_feature.__name__, time_feature(timestamp_as_index))\n",
    "    for time_feature in time_features\n",
    "]\n",
    "print(dict(additional_features))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bd7b8b9",
   "metadata": {},
   "source": [
    "Note that hours and days are encoded as values between `[-0.5, 0.5]` from GluonTS. For more information about `time_features`, please see [this](https://github.com/awslabs/gluonts/blob/dev/src/gluonts/time_feature/_base.py). Besides those 4 features, we'll also add an \"age\" feature as we'll see later on in the data transformations.\n",
    "\n",
    "We now have everything to define the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3dda0e78",
   "metadata": {
    "id": "3dda0e78"
   },
   "outputs": [],
   "source": [
    "from transformers import InformerConfig, InformerForPrediction\n",
    "\n",
    "config = InformerConfig(\n",
    "    # in the multivariate setting, input_size is the number of variates in the time series per time step\n",
    "    input_size=num_of_variates,\n",
    "    # prediction length:\n",
    "    prediction_length=prediction_length,\n",
    "    # context length:\n",
    "    context_length=prediction_length * 2,\n",
    "    # lags value copied from 1 week before:\n",
    "    lags_sequence=[1, 24 * 7],\n",
    "    # we'll add 5 time features (\"hour_of_day\", ..., and \"age\"):\n",
    "    num_time_features=len(time_features) + 1,\n",
    "    # informer params:\n",
    "    dropout=0.1,\n",
    "    encoder_layers=6,\n",
    "    decoder_layers=4,\n",
    "    # project input from num_of_variates*len(lags_sequence)+num_time_features to:\n",
    "    d_model=64,\n",
    ")\n",
    "\n",
    "model = InformerForPrediction(config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "T6rCeV4dsUnF",
   "metadata": {
    "id": "T6rCeV4dsUnF"
   },
   "source": [
    "By default, the model uses a diagonal Student-t distribution (but this is [configurable](https://huggingface.co/docs/transformers/main/en/internal/time_series_utils)):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "EaoKZyujsuIA",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "EaoKZyujsuIA",
    "outputId": "e711cbea-c734-463f-f28c-4009e5317a4e"
   },
   "outputs": [],
   "source": [
    "model.config.distribution_output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af82c60d",
   "metadata": {
    "id": "af82c60d"
   },
   "source": [
    "## Define Transformations\n",
    "\n",
    "Next, we define the transformations for the data, in particular for the creation of the time features (based on the dataset or universal ones).\n",
    "\n",
    "Again, we'll use the GluonTS library for this. We define a `Chain` of transformations (which is a bit comparable to `torchvision.transforms.Compose` for images). It allows us to combine several transformations into a single pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "tR87yKPSn8SZ",
   "metadata": {
    "id": "tR87yKPSn8SZ"
   },
   "outputs": [],
   "source": [
    "from gluonts.time_feature import TimeFeature\n",
    "from gluonts.dataset.field_names import FieldName\n",
    "from gluonts.transform import (\n",
    "    AddAgeFeature,\n",
    "    AddObservedValuesIndicator,\n",
    "    AddTimeFeatures,\n",
    "    AsNumpyArray,\n",
    "    Chain,\n",
    "    ExpectedNumInstanceSampler,\n",
    "    InstanceSplitter,\n",
    "    RemoveFields,\n",
    "    SelectFields,\n",
    "    SetField,\n",
    "    TestSplitSampler,\n",
    "    Transformation,\n",
    "    ValidationSplitSampler,\n",
    "    VstackFeatures,\n",
    "    RenameFields,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4Qql4CInFWO7",
   "metadata": {
    "id": "4Qql4CInFWO7"
   },
   "source": [
    "The transformations below are annotated with comments, to explain what they do. At a high level, we will iterate over the individual time series of our dataset and add/remove fields or features:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20fe036e",
   "metadata": {
    "id": "20fe036e"
   },
   "outputs": [],
   "source": [
    "from transformers import PretrainedConfig\n",
    "\n",
    "\n",
    "def create_transformation(freq: str, config: PretrainedConfig) -> Transformation:\n",
    "    # create list of fields to remove later\n",
    "    remove_field_names = []\n",
    "    if config.num_static_real_features == 0:\n",
    "        remove_field_names.append(FieldName.FEAT_STATIC_REAL)\n",
    "    if config.num_dynamic_real_features == 0:\n",
    "        remove_field_names.append(FieldName.FEAT_DYNAMIC_REAL)\n",
    "    if config.num_static_categorical_features == 0:\n",
    "        remove_field_names.append(FieldName.FEAT_STATIC_CAT)\n",
    "\n",
    "    return Chain(\n",
    "        # step 1: remove static/dynamic fields if not specified\n",
    "        [RemoveFields(field_names=remove_field_names)]\n",
    "        # step 2: convert the data to NumPy (potentially not needed)\n",
    "        + (\n",
    "            [\n",
    "                AsNumpyArray(\n",
    "                    field=FieldName.FEAT_STATIC_CAT,\n",
    "                    expected_ndim=1,\n",
    "                    dtype=int,\n",
    "                )\n",
    "            ]\n",
    "            if config.num_static_categorical_features > 0\n",
    "            else []\n",
    "        )\n",
    "        + (\n",
    "            [\n",
    "                AsNumpyArray(\n",
    "                    field=FieldName.FEAT_STATIC_REAL,\n",
    "                    expected_ndim=1,\n",
    "                )\n",
    "            ]\n",
    "            if config.num_static_real_features > 0\n",
    "            else []\n",
    "        )\n",
    "        + [\n",
    "            AsNumpyArray(\n",
    "                field=FieldName.TARGET,\n",
    "                # we expect an extra dim for the multivariate case:\n",
    "                expected_ndim=1 if config.input_size == 1 else 2,\n",
    "            ),\n",
    "            # step 3: handle the NaN's by filling in the target with zero\n",
    "            # and return the mask (which is in the observed values)\n",
    "            # true for observed values, false for nan's\n",
    "            # the decoder uses this mask (no loss is incurred for unobserved values)\n",
    "            # see loss_weights inside the xxxForPrediction model\n",
    "            AddObservedValuesIndicator(\n",
    "                target_field=FieldName.TARGET,\n",
    "                output_field=FieldName.OBSERVED_VALUES,\n",
    "            ),\n",
    "            # step 4: add temporal features based on freq of the dataset\n",
    "            # these serve as positional encodings\n",
    "            AddTimeFeatures(\n",
    "                start_field=FieldName.START,\n",
    "                target_field=FieldName.TARGET,\n",
    "                output_field=FieldName.FEAT_TIME,\n",
    "                time_features=time_features_from_frequency_str(freq),\n",
    "                pred_length=config.prediction_length,\n",
    "            ),\n",
    "            # step 5: add another temporal feature (just a single number)\n",
    "            # tells the model where in the life the value of the time series is\n",
    "            # sort of running counter\n",
    "            AddAgeFeature(\n",
    "                target_field=FieldName.TARGET,\n",
    "                output_field=FieldName.FEAT_AGE,\n",
    "                pred_length=config.prediction_length,\n",
    "                log_scale=True,\n",
    "            ),\n",
    "            # step 6: vertically stack all the temporal features into the key FEAT_TIME\n",
    "            VstackFeatures(\n",
    "                output_field=FieldName.FEAT_TIME,\n",
    "                input_fields=[FieldName.FEAT_TIME, FieldName.FEAT_AGE]\n",
    "                + (\n",
    "                    [FieldName.FEAT_DYNAMIC_REAL]\n",
    "                    if config.num_dynamic_real_features > 0\n",
    "                    else []\n",
    "                ),\n",
    "            ),\n",
    "            # step 7: rename to match HuggingFace names\n",
    "            RenameFields(\n",
    "                mapping={\n",
    "                    FieldName.FEAT_STATIC_CAT: \"static_categorical_features\",\n",
    "                    FieldName.FEAT_STATIC_REAL: \"static_real_features\",\n",
    "                    FieldName.FEAT_TIME: \"time_features\",\n",
    "                    FieldName.TARGET: \"values\",\n",
    "                    FieldName.OBSERVED_VALUES: \"observed_mask\",\n",
    "                }\n",
    "            ),\n",
    "        ]\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bab46d0",
   "metadata": {
    "id": "8bab46d0"
   },
   "source": [
    "## Define `InstanceSplitter`\n",
    "\n",
    "For training/validation/testing we next create an `InstanceSplitter` which is used to sample windows from the dataset (as, remember, we can't pass the entire history of values to the model due to time- and memory constraints).\n",
    "\n",
    "The instance splitter samples random `context_length` sized and subsequent `prediction_length` sized windows from the data, and appends a `past_` or `future_` key to any temporal keys in `time_series_fields` for the respective windows. The instance splitter can be configured into three different modes:\n",
    "1. `mode=\"train\"`: Here we sample the context and prediction length windows randomly from the dataset given to it (the training dataset)\n",
    "2. `mode=\"validation\"`: Here we sample the very last context length window and prediction window from the dataset given to it (for the back-testing or validation likelihood calculations)\n",
    "3. `mode=\"test\"`: Here we sample the very last context length window only (for the prediction use case)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cae7600d",
   "metadata": {
    "id": "cae7600d"
   },
   "outputs": [],
   "source": [
    "from gluonts.transform.sampler import InstanceSampler\n",
    "from typing import Optional\n",
    "\n",
    "\n",
    "def create_instance_splitter(\n",
    "    config: PretrainedConfig,\n",
    "    mode: str,\n",
    "    train_sampler: Optional[InstanceSampler] = None,\n",
    "    validation_sampler: Optional[InstanceSampler] = None,\n",
    ") -> Transformation:\n",
    "    assert mode in [\"train\", \"validation\", \"test\"]\n",
    "\n",
    "    instance_sampler = {\n",
    "        \"train\": train_sampler\n",
    "        or ExpectedNumInstanceSampler(\n",
    "            num_instances=1.0, min_future=config.prediction_length\n",
    "        ),\n",
    "        \"validation\": validation_sampler\n",
    "        or ValidationSplitSampler(min_future=config.prediction_length),\n",
    "        \"test\": TestSplitSampler(),\n",
    "    }[mode]\n",
    "\n",
    "    return InstanceSplitter(\n",
    "        target_field=\"values\",\n",
    "        is_pad_field=FieldName.IS_PAD,\n",
    "        start_field=FieldName.START,\n",
    "        forecast_start_field=FieldName.FORECAST_START,\n",
    "        instance_sampler=instance_sampler,\n",
    "        past_length=config.context_length + max(config.lags_sequence),\n",
    "        future_length=config.prediction_length,\n",
    "        time_series_fields=[\"time_features\", \"observed_mask\"],\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "958e050d",
   "metadata": {
    "id": "958e050d"
   },
   "source": [
    "## Create DataLoaders\n",
    "\n",
    "Next, it's time to create the DataLoaders, which allow us to have batches of (input, output) pairs - or in other words (`past_values`, `future_values`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6995101c",
   "metadata": {
    "id": "6995101c"
   },
   "outputs": [],
   "source": [
    "from typing import Iterable\n",
    "\n",
    "import torch\n",
    "from gluonts.itertools import Cached, Cyclic\n",
    "from gluonts.dataset.loader import as_stacked_batches\n",
    "\n",
    "\n",
    "def create_train_dataloader(\n",
    "    config: PretrainedConfig,\n",
    "    freq,\n",
    "    data,\n",
    "    batch_size: int,\n",
    "    num_batches_per_epoch: int,\n",
    "    shuffle_buffer_length: Optional[int] = None,\n",
    "    cache_data: bool = True,\n",
    "    **kwargs,\n",
    ") -> Iterable:\n",
    "    PREDICTION_INPUT_NAMES = [\n",
    "        \"past_time_features\",\n",
    "        \"past_values\",\n",
    "        \"past_observed_mask\",\n",
    "        \"future_time_features\",\n",
    "    ]\n",
    "    if config.num_static_categorical_features > 0:\n",
    "        PREDICTION_INPUT_NAMES.append(\"static_categorical_features\")\n",
    "\n",
    "    if config.num_static_real_features > 0:\n",
    "        PREDICTION_INPUT_NAMES.append(\"static_real_features\")\n",
    "\n",
    "    TRAINING_INPUT_NAMES = PREDICTION_INPUT_NAMES + [\n",
    "        \"future_values\",\n",
    "        \"future_observed_mask\",\n",
    "    ]\n",
    "\n",
    "    transformation = create_transformation(freq, config)\n",
    "    transformed_data = transformation.apply(data, is_train=True)\n",
    "    if cache_data:\n",
    "        transformed_data = Cached(transformed_data)\n",
    "\n",
    "    # we initialize a Training instance\n",
    "    instance_splitter = create_instance_splitter(config, \"train\")\n",
    "\n",
    "    # the instance splitter will sample a window of\n",
    "    # context length + lags + prediction length (from all the possible transformed time series, 1 in our case)\n",
    "    # randomly from within the target time series and return an iterator.\n",
    "    stream = Cyclic(transformed_data).stream()\n",
    "    training_instances = instance_splitter.apply(stream)\n",
    "\n",
    "    return as_stacked_batches(\n",
    "        training_instances,\n",
    "        batch_size=batch_size,\n",
    "        shuffle_buffer_length=shuffle_buffer_length,\n",
    "        field_names=TRAINING_INPUT_NAMES,\n",
    "        output_type=torch.tensor,\n",
    "        num_batches_per_epoch=num_batches_per_epoch,\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10c55455",
   "metadata": {
    "id": "10c55455"
   },
   "outputs": [],
   "source": [
    "def create_backtest_dataloader(\n",
    "    config: PretrainedConfig,\n",
    "    freq,\n",
    "    data,\n",
    "    batch_size: int,\n",
    "    **kwargs,\n",
    "):\n",
    "    PREDICTION_INPUT_NAMES = [\n",
    "        \"past_time_features\",\n",
    "        \"past_values\",\n",
    "        \"past_observed_mask\",\n",
    "        \"future_time_features\",\n",
    "    ]\n",
    "    if config.num_static_categorical_features > 0:\n",
    "        PREDICTION_INPUT_NAMES.append(\"static_categorical_features\")\n",
    "\n",
    "    if config.num_static_real_features > 0:\n",
    "        PREDICTION_INPUT_NAMES.append(\"static_real_features\")\n",
    "\n",
    "    transformation = create_transformation(freq, config)\n",
    "    transformed_data = transformation.apply(data)\n",
    "\n",
    "    # we create a Validation Instance splitter which will sample the very last\n",
    "    # context window seen during training only for the encoder.\n",
    "    instance_sampler = create_instance_splitter(config, \"validation\")\n",
    "\n",
    "    # we apply the transformations in train mode\n",
    "    testing_instances = instance_sampler.apply(transformed_data, is_train=True)\n",
    "\n",
    "    return as_stacked_batches(\n",
    "        testing_instances,\n",
    "        batch_size=batch_size,\n",
    "        output_type=torch.tensor,\n",
    "        field_names=PREDICTION_INPUT_NAMES,\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "997e43dd-3861-452b-98c5-b5ca3c7da4af",
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_test_dataloader(\n",
    "    config: PretrainedConfig,\n",
    "    freq,\n",
    "    data,\n",
    "    batch_size: int,\n",
    "    **kwargs,\n",
    "):\n",
    "    PREDICTION_INPUT_NAMES = [\n",
    "        \"past_time_features\",\n",
    "        \"past_values\",\n",
    "        \"past_observed_mask\",\n",
    "        \"future_time_features\",\n",
    "    ]\n",
    "    if config.num_static_categorical_features > 0:\n",
    "        PREDICTION_INPUT_NAMES.append(\"static_categorical_features\")\n",
    "\n",
    "    if config.num_static_real_features > 0:\n",
    "        PREDICTION_INPUT_NAMES.append(\"static_real_features\")\n",
    "\n",
    "    transformation = create_transformation(freq, config)\n",
    "    transformed_data = transformation.apply(data, is_train=False)\n",
    "\n",
    "    # We create a test Instance splitter to sample the very last\n",
    "    # context window from the dataset provided.\n",
    "    instance_sampler = create_instance_splitter(config, \"test\")\n",
    "\n",
    "    # We apply the transformations in test mode\n",
    "    testing_instances = instance_sampler.apply(transformed_data, is_train=False)\n",
    "\n",
    "    return as_stacked_batches(\n",
    "        testing_instances,\n",
    "        batch_size=batch_size,\n",
    "        output_type=torch.tensor,\n",
    "        field_names=PREDICTION_INPUT_NAMES,\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20e2338b",
   "metadata": {
    "id": "20e2338b"
   },
   "outputs": [],
   "source": [
    "train_dataloader = create_train_dataloader(\n",
    "    config=config,\n",
    "    freq=freq,\n",
    "    data=multi_variate_train_dataset,\n",
    "    batch_size=256,\n",
    "    num_batches_per_epoch=100,\n",
    "    num_workers=2,\n",
    ")\n",
    "\n",
    "test_dataloader = create_backtest_dataloader(\n",
    "    config=config,\n",
    "    freq=freq,\n",
    "    data=multi_variate_test_dataset,\n",
    "    batch_size=32,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ES2U8g-2G2Jd",
   "metadata": {
    "id": "ES2U8g-2G2Jd"
   },
   "source": [
    "Let's check the first batch:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "YU2h9OOB5IsX",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YU2h9OOB5IsX",
    "outputId": "ba8073bb-f450-4cd1-faee-3fb129f35c7e"
   },
   "outputs": [],
   "source": [
    "batch = next(iter(train_dataloader))\n",
    "for k, v in batch.items():\n",
    "    print(k, v.shape, v.type())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "HvvPlSF8HBYd",
   "metadata": {
    "id": "HvvPlSF8HBYd"
   },
   "source": [
    "As can be seen, we don't feed `input_ids` and `attention_mask` to the encoder (as would be the case for NLP models), but rather `past_values`, along with `past_observed_mask`, `past_time_features` and `static_real_features`.\n",
    "\n",
    "The decoder inputs consist of `future_values`, `future_observed_mask` and `future_time_features`. The `future_values` can be seen as the equivalent of `decoder_input_ids` in NLP.\n",
    "\n",
    "We refer to the [docs](https://huggingface.co/docs/transformers/main/en/model_doc/informer#transformers.InformerForPrediction.forward.past_values) for a detailed explanation for each of them."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "_cev4ufVv1yf",
   "metadata": {
    "id": "_cev4ufVv1yf"
   },
   "source": [
    "## Forward pass\n",
    "\n",
    "Let's perform a single forward pass with the batch we just created:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "sD8fav6qTApR",
   "metadata": {
    "id": "sD8fav6qTApR"
   },
   "outputs": [],
   "source": [
    "# perform forward pass\n",
    "outputs = model(\n",
    "    past_values=batch[\"past_values\"],\n",
    "    past_time_features=batch[\"past_time_features\"],\n",
    "    past_observed_mask=batch[\"past_observed_mask\"],\n",
    "    static_categorical_features=batch[\"static_categorical_features\"]\n",
    "    if config.num_static_categorical_features > 0\n",
    "    else None,\n",
    "    static_real_features=batch[\"static_real_features\"]\n",
    "    if config.num_static_real_features > 0\n",
    "    else None,\n",
    "    future_values=batch[\"future_values\"],\n",
    "    future_time_features=batch[\"future_time_features\"],\n",
    "    future_observed_mask=batch[\"future_observed_mask\"],\n",
    "    output_hidden_states=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "j2KnnHTCX4RC",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "j2KnnHTCX4RC",
    "outputId": "7b11e3b4-c4e1-48fd-c93e-69a21f28e072"
   },
   "outputs": [],
   "source": [
    "print(\"Loss:\", outputs.loss.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "V9K8s2j9y8x7",
   "metadata": {
    "id": "V9K8s2j9y8x7"
   },
   "source": [
    "Note that the model is returning a loss. This is possible as the decoder automatically shifts the `future_values` one position to the right in order to have the labels. This allows computing a loss between the predicted values and the labels. The loss is the negative log-likelihood of the predicted distribution with respect to the ground truth values and tends to negative infinity.\n",
    "\n",
    "Also note that the decoder uses a causal mask to not look into the future as the values it needs to predict are in the `future_values` tensor."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "SxHDCa7vwPBF",
   "metadata": {
    "id": "SxHDCa7vwPBF"
   },
   "source": [
    "## Train the Model\n",
    "\n",
    "It's time to train the model! We'll use a standard PyTorch training loop.\n",
    "\n",
    "We will use the 🤗 [Accelerate](https://huggingface.co/docs/accelerate/index) library here, which automatically places the model, optimizer and dataloader on the appropriate `device`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "gMLYvQaNHuXQ",
   "metadata": {
    "id": "gMLYvQaNHuXQ"
   },
   "outputs": [],
   "source": [
    "from accelerate import Accelerator\n",
    "from torch.optim import AdamW\n",
    "\n",
    "epochs = 25\n",
    "loss_history = []\n",
    "\n",
    "accelerator = Accelerator()\n",
    "device = accelerator.device\n",
    "\n",
    "model.to(device)\n",
    "optimizer = AdamW(model.parameters(), lr=6e-4, betas=(0.9, 0.95), weight_decay=1e-1)\n",
    "\n",
    "model, optimizer, train_dataloader = accelerator.prepare(\n",
    "    model,\n",
    "    optimizer,\n",
    "    train_dataloader,\n",
    ")\n",
    "\n",
    "model.train()\n",
    "for epoch in range(epochs):\n",
    "    for idx, batch in enumerate(train_dataloader):\n",
    "        optimizer.zero_grad()\n",
    "        outputs = model(\n",
    "            static_categorical_features=batch[\"static_categorical_features\"].to(device)\n",
    "            if config.num_static_categorical_features > 0\n",
    "            else None,\n",
    "            static_real_features=batch[\"static_real_features\"].to(device)\n",
    "            if config.num_static_real_features > 0\n",
    "            else None,\n",
    "            past_time_features=batch[\"past_time_features\"].to(device),\n",
    "            past_values=batch[\"past_values\"].to(device),\n",
    "            future_time_features=batch[\"future_time_features\"].to(device),\n",
    "            future_values=batch[\"future_values\"].to(device),\n",
    "            past_observed_mask=batch[\"past_observed_mask\"].to(device),\n",
    "            future_observed_mask=batch[\"future_observed_mask\"].to(device),\n",
    "        )\n",
    "        loss = outputs.loss\n",
    "\n",
    "        # Backpropagation\n",
    "        accelerator.backward(loss)\n",
    "        optimizer.step()\n",
    "\n",
    "        loss_history.append(loss.item())\n",
    "        if idx % 100 == 0:\n",
    "            print(loss.item())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb1e9bb8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# view training\n",
    "loss_history = np.array(loss_history).reshape(-1)\n",
    "x = range(loss_history.shape[0])\n",
    "plt.figure(figsize=(10, 5))\n",
    "plt.plot(x, loss_history, label=\"train\")\n",
    "plt.title(\"Loss\", fontsize=15)\n",
    "plt.legend(loc=\"upper right\")\n",
    "plt.xlabel(\"iteration\")\n",
    "plt.ylabel(\"nll\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Q1U6YhaFXlSp",
   "metadata": {
    "id": "Q1U6YhaFXlSp"
   },
   "source": [
    "## Inference\n",
    "\n",
    "At inference time, it's recommended to use the `generate()` method for autoregressive generation, similar to NLP models.\n",
    "\n",
    "Forecasting involves getting data from the test instance sampler, which will sample the very last `context_length` sized window of values from each time series in the dataset, and pass it to the model. Note that we pass `future_time_features`, which are known ahead of time, to the decoder.\n",
    "\n",
    "The model will autoregressively sample a certain number of values from the predicted distribution and pass them back to the decoder to return the prediction outputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c7482c1",
   "metadata": {
    "id": "7c7482c1"
   },
   "outputs": [],
   "source": [
    "model.eval()\n",
    "\n",
    "forecasts_ = []\n",
    "\n",
    "for batch in test_dataloader:\n",
    "    outputs = model.generate(\n",
    "        static_categorical_features=batch[\"static_categorical_features\"].to(device)\n",
    "        if config.num_static_categorical_features > 0\n",
    "        else None,\n",
    "        static_real_features=batch[\"static_real_features\"].to(device)\n",
    "        if config.num_static_real_features > 0\n",
    "        else None,\n",
    "        past_time_features=batch[\"past_time_features\"].to(device),\n",
    "        past_values=batch[\"past_values\"].to(device),\n",
    "        future_time_features=batch[\"future_time_features\"].to(device),\n",
    "        past_observed_mask=batch[\"past_observed_mask\"].to(device),\n",
    "    )\n",
    "    forecasts_.append(outputs.sequences.cpu().numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "kPLiRcOeZR67",
   "metadata": {
    "id": "kPLiRcOeZR67"
   },
   "source": [
    "The model outputs a tensor of shape (`batch_size`, `number of samples`, `prediction length`, `input_size`). \n",
    "\n",
    "In this case, we get `100` possible values for the next `48` hours for each of the `862` time series (for each example in the batch which is of size `1` since we only have a single multivariate time series):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "DwAfSZitZNAQ",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "DwAfSZitZNAQ",
    "outputId": "8ca71318-2361-4c9c-ae93-88ba9a453fef"
   },
   "outputs": [],
   "source": [
    "forecasts_[0].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fCTBw_t91xwH",
   "metadata": {
    "id": "fCTBw_t91xwH"
   },
   "source": [
    "We'll stack them vertically, to get forecasts for all time-series in the test dataset (just in case there are more time series in the test set):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "du1GyJVXlpHp",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "du1GyJVXlpHp",
    "outputId": "cfd3e777-da77-4ca6-eedd-e0d0f22da872"
   },
   "outputs": [],
   "source": [
    "forecasts = np.vstack(forecasts_)\n",
    "print(forecasts.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wlvFCdgiA9oy",
   "metadata": {
    "id": "wlvFCdgiA9oy"
   },
   "source": [
    "We can evaluate the resulting forecast with respect to the ground truth out of sample values present in the test set. For that, we'll use the 🤗 [Evaluate](https://huggingface.co/docs/evaluate/index) library, which includes the [MASE](https://huggingface.co/spaces/evaluate-metric/mase) and [sMAPE](https://huggingface.co/spaces/evaluate-metric/smape) metrics.\n",
    "\n",
    "We calculate both metrics for each time series variate in the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0yb9RnczYE4z",
   "metadata": {
    "id": "0yb9RnczYE4z"
   },
   "outputs": [],
   "source": [
    "from evaluate import load\n",
    "from gluonts.time_feature import get_seasonality\n",
    "\n",
    "mase_metric = load(\"evaluate-metric/mase\")\n",
    "smape_metric = load(\"evaluate-metric/smape\")\n",
    "\n",
    "forecast_median = np.median(forecasts, 1).squeeze(0).T\n",
    "\n",
    "mase_metrics = []\n",
    "smape_metrics = []\n",
    "\n",
    "for item_id, ts in enumerate(test_dataset):\n",
    "    training_data = ts[\"target\"][:-prediction_length]\n",
    "    ground_truth = ts[\"target\"][-prediction_length:]\n",
    "    mase = mase_metric.compute(\n",
    "        predictions=forecast_median[item_id],\n",
    "        references=np.array(ground_truth),\n",
    "        training=np.array(training_data),\n",
    "        periodicity=get_seasonality(freq),\n",
    "    )\n",
    "    mase_metrics.append(mase[\"mase\"])\n",
    "\n",
    "    smape = smape_metric.compute(\n",
    "        predictions=forecast_median[item_id],\n",
    "        references=np.array(ground_truth),\n",
    "    )\n",
    "    smape_metrics.append(smape[\"smape\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fuuFbNYdZlIR",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fuuFbNYdZlIR",
    "outputId": "9a010645-30fc-4af8-bc53-636391ba0ec6"
   },
   "outputs": [],
   "source": [
    "print(f\"MASE: {np.mean(mase_metrics)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "w6ETpUrML2wE",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "w6ETpUrML2wE",
    "outputId": "77127e13-36b1-4cd2-9f90-63f3f23c6db0"
   },
   "outputs": [],
   "source": [
    "print(f\"sMAPE: {np.mean(smape_metrics)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b649b65e",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.scatter(mase_metrics, smape_metrics, alpha=0.2)\n",
    "plt.xlabel(\"MASE\")\n",
    "plt.ylabel(\"sMAPE\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "moksM2QmMACr",
   "metadata": {
    "id": "moksM2QmMACr"
   },
   "source": [
    "To plot the prediction for any time series variate with respect the ground truth test data we define the following helper:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4e35b93e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.dates as mdates\n",
    "\n",
    "\n",
    "def plot(ts_index, mv_index):\n",
    "    fig, ax = plt.subplots()\n",
    "\n",
    "    index = pd.period_range(\n",
    "        start=multi_variate_test_dataset[ts_index][FieldName.START],\n",
    "        periods=len(multi_variate_test_dataset[ts_index][FieldName.TARGET]),\n",
    "        freq=multi_variate_test_dataset[ts_index][FieldName.START].freq,\n",
    "    ).to_timestamp()\n",
    "\n",
    "    ax.xaxis.set_minor_locator(mdates.HourLocator())\n",
    "\n",
    "    ax.plot(\n",
    "        index[-2 * prediction_length :],\n",
    "        multi_variate_test_dataset[ts_index][\"target\"][\n",
    "            mv_index, -2 * prediction_length :\n",
    "        ],\n",
    "        label=\"actual\",\n",
    "    )\n",
    "\n",
    "    ax.plot(\n",
    "        index[-prediction_length:],\n",
    "        forecasts[ts_index, ..., mv_index].mean(axis=0),\n",
    "        label=\"mean\",\n",
    "    )\n",
    "    ax.fill_between(\n",
    "        index[-prediction_length:],\n",
    "        forecasts[ts_index, ..., mv_index].mean(0)\n",
    "        - forecasts[ts_index, ..., mv_index].std(axis=0),\n",
    "        forecasts[ts_index, ..., mv_index].mean(0)\n",
    "        + forecasts[ts_index, ..., mv_index].std(axis=0),\n",
    "        alpha=0.2,\n",
    "        interpolate=True,\n",
    "        label=\"+/- 1-std\",\n",
    "    )\n",
    "    ax.legend()\n",
    "    fig.autofmt_xdate()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mwtksAcxMHoK",
   "metadata": {
    "id": "mwtksAcxMHoK"
   },
   "source": [
    "For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5N8fdnm_MKQP",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 265
    },
    "id": "5N8fdnm_MKQP",
    "outputId": "3b221c31-ee20-4626-9592-2492ce143b4a"
   },
   "outputs": [],
   "source": [
    "plot(0, 344)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Nf4Y8MvLMOi8",
   "metadata": {
    "id": "Nf4Y8MvLMOi8"
   },
   "source": [
    "## Conclusion\n",
    "\n",
    "How do we compare against other models? The [Monash Time Series Repository](https://forecastingdata.org/#results) has a comparison table of test set MASE metrics which we can add to:\n",
    "\n",
    "|Dataset | \tSES| \tTheta | \tTBATS| \tETS\t| (DHR-)ARIMA| \tPR|\tCatBoost |\tFFNN\t| DeepAR | \tN-BEATS | \tWaveNet|  Transformer (uni.) | **Informer (mv. our)**| \n",
    "|:------------------:|:-----------------:|:--:|:--:|:--:|:--:|:--:|:--:|:---:|:---:|:--:|:--:|:--:|:--:|\n",
    "|Traffic Hourly | 1.922\t| 1.922\t| 2.482 |\t2.294|\t2.535|\t1.281|\t1.571\t|0.892|\t0.825\t|1.100|\t1.066\t| **0.821** | 1.191 |\n",
    "\n",
    "As can be seen, and perhaps surprising to some, the multivariate forecasts are typically _worse_ than the univariate ones, the reason being the difficulty in estimating the cross-series correlations/relationships. The additional variance added by the estimates often harms the resulting forecasts or the model learns spurious correlations. We refer to [this paper](https://openreview.net/forum?id=GpW327gxLTF) for further reading. Multivariate models tend to work well when trained on a lot of data.\n",
    "\n",
    "So the vanilla Transformer still performs best here! In the future, we hope to better benchmark these models in a central place to ease reproducing the results of several papers. Stay tuned for more!\n",
    "\n",
    "## Resources\n",
    "\n",
    "We recommend to check out the [Informer docs](https://huggingface.co/docs/transformers/main/en/model_doc/informer) and the example notebook linked at the top of this blog post."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec04e604",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "include_colab_link": true,
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "009a21c6a2dd4c5da79a7fb3eec8ff43": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a8b516bf196946a09a55732d937dfe2a",
      "placeholder": "​",
      "style": "IPY_MODEL_53a3ad2f991648439e57362c4a7a5f60",
      "value": "Generating test split:  89%"
     }
    },
    "01a6160718434f7bb73bb4a094ee8b0f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "028908857f134943af7b47a2503ce403": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_50e91b016b0a4ce69325e8196a09dce6",
      "placeholder": "​",
      "style": "IPY_MODEL_9c573dc9c14248deb512162af5f9b0fa",
      "value": " 326/366 [00:00&lt;00:00, 615.43 examples/s]"
     }
    },
    "044030ba2c5948d2ac6545aea773ef4a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0452728577c4433d9d15eca97045d1f4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "048591e682544b57abbcba613c81f9b4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_d8a6361f19894be1b730137b0801e4e8",
       "IPY_MODEL_2c375a08b1cb4f71a0a0a169c999aa94",
       "IPY_MODEL_ea1af7f963e848d19ea363331027109b"
      ],
      "layout": "IPY_MODEL_b68b7b7b072e457980e834113ff0a670"
     }
    },
    "09cc068788e64c1fbc1edd0992f43f42": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0a4bcc907dbc48f3af39fac4c0652bd4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0bf375322d12400ab773e85c08cd3e62": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_22c8c8ecb39c491798602abe0144eba1",
      "placeholder": "​",
      "style": "IPY_MODEL_36a6c51643ba4375b314e9395c0aea07",
      "value": " 311/366 [00:01&lt;00:00, 436.98 examples/s]"
     }
    },
    "0d037e1f14794a0fb68ab472b42ee60a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_eaa70494878f4eba927868ecbbd034b4",
       "IPY_MODEL_b9f43527572c41849f66e100e4d2df83",
       "IPY_MODEL_f0019a4884124eebb44571ca0a7b262e"
      ],
      "layout": "IPY_MODEL_a79295d7dc77434282636a0ef665a70b"
     }
    },
    "1076c8e3f8e1413fad4aa5cb16c71c31": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_dbb568c5b348421a9e2f9f4e07cd5f01",
       "IPY_MODEL_966ca413bd824a77b50a20784e8bd1b4",
       "IPY_MODEL_e81a3f25949247eeb696ccd3b0df3da0"
      ],
      "layout": "IPY_MODEL_228e4cf452a04b1f8f324be138857c30"
     }
    },
    "17a314c74019462e9ba2e923b9235bed": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6b97acade12a49d683ec81855e1bf58a",
      "placeholder": "​",
      "style": "IPY_MODEL_45b1dbee32a440328f91bf0c398d4a88",
      "value": " 346/366 [00:00&lt;00:00, 663.14 examples/s]"
     }
    },
    "228e4cf452a04b1f8f324be138857c30": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "22c8c8ecb39c491798602abe0144eba1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "23207dda3c7c45ef942fb26b6d3e8f4a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2435607c37f2439aa1f0d545009b375e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": "hidden",
      "width": null
     }
    },
    "2c375a08b1cb4f71a0a0a169c999aa94": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a3384254326f49eb907bfa6d2032b1a8",
      "max": 3,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_68e542f3c052435f9254e135e5c481e8",
      "value": 3
     }
    },
    "2dc9cef88370404184dc1f629d062a04": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ed58c5b583f344cdbfea76562259190e",
       "IPY_MODEL_c56009b340984f2296ad2959e8d72f25",
       "IPY_MODEL_b2eeaecdd52343a4977954915f38bc9b"
      ],
      "layout": "IPY_MODEL_c55cb9a98afe49d59838a3e2d88cd774"
     }
    },
    "2e0511f2f1844758a6f6d10af7233a72": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "2ec6b12a621f4ce681554397392bdda8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6a39d27a5d5e484385f3d8163665641f",
      "placeholder": "​",
      "style": "IPY_MODEL_649640a4dacf4edab3aba273d7e09bbf",
      "value": " 31.2k/31.2k [00:00&lt;00:00, 18.6kB/s]"
     }
    },
    "313d214425c541b6845f70638f8a5937": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "331752b5913245e380752532ab15f128": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3366b16d73d949508f24d647558db059": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "36a6c51643ba4375b314e9395c0aea07": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3c850c33752443afa6658a1cb997549e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3e1c8ac5bfac429993d5404f7abf748d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "3ff0a924499f414ca7fd03a902b51102": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "435416a295674774a15aa700ff7ce007": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "43daab71788e4a5c80687a9e73bf3b67": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_67f8fc82c7044090a0408294bc1c2d15",
       "IPY_MODEL_7cd0096ebb94429dab18e3226fc4cdbf",
       "IPY_MODEL_0bf375322d12400ab773e85c08cd3e62"
      ],
      "layout": "IPY_MODEL_e3641728ace74d8ab4742007d1725ee8"
     }
    },
    "445f2ef3f71a4573ae453144dd4c5e81": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": "hidden",
      "width": null
     }
    },
    "45b1dbee32a440328f91bf0c398d4a88": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "46e05d2996a84b5e8f456adbb6453350": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "50e91b016b0a4ce69325e8196a09dce6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "53938941490b4afea31c63aad4bdce0a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "53a3ad2f991648439e57362c4a7a5f60": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5c531b5b3ce4406abde898297f4bc80b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b7e131bcb6454ccbb5de0813a734586a",
      "max": 24887,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_3e1c8ac5bfac429993d5404f7abf748d",
      "value": 24887
     }
    },
    "630d2a4aba53498b8e6f40d713fe50d1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "649640a4dacf4edab3aba273d7e09bbf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "672ee2ad96ac42fa90645a4049c7dd16": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "67f8fc82c7044090a0408294bc1c2d15": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3c850c33752443afa6658a1cb997549e",
      "placeholder": "​",
      "style": "IPY_MODEL_3366b16d73d949508f24d647558db059",
      "value": "Generating train split:  85%"
     }
    },
    "682be41e57c341189bdc3e7a6b7246d3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "68e542f3c052435f9254e135e5c481e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "6a39d27a5d5e484385f3d8163665641f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6b97acade12a49d683ec81855e1bf58a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "726309e90e0247628487e1f4816e6fd9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "741afc0fb9a7469cac4a1d6b0dc130d8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "764abccaae30410ba68405f44875c8d6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7bb9e900006840a7a892a1a48d676560": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "7cc6a3332eaf4e9bb012097ca03c6ef6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7cd0096ebb94429dab18e3226fc4cdbf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_741afc0fb9a7469cac4a1d6b0dc130d8",
      "max": 366,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_df02d3127e634862b1029787be8908b8",
      "value": 366
     }
    },
    "83c39de970e84954b6fa043390b85d4c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_009a21c6a2dd4c5da79a7fb3eec8ff43",
       "IPY_MODEL_9059a60d370c4825a0eb1b31c9f9cd0a",
       "IPY_MODEL_028908857f134943af7b47a2503ce403"
      ],
      "layout": "IPY_MODEL_445f2ef3f71a4573ae453144dd4c5e81"
     }
    },
    "87d63881c376462b88a7703f62495a83": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_af5add21a56b4ef4945a90fed06d7c89",
       "IPY_MODEL_c0822d9ca57b4b23ba552b93f139d96e",
       "IPY_MODEL_2ec6b12a621f4ce681554397392bdda8"
      ],
      "layout": "IPY_MODEL_8de92b21ea1b4dfc966d653e0ca7043b"
     }
    },
    "89dd9177f7cd4edc87e43d5e23c5f587": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8b30e8b778fb456bbf8af4d6931f415c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_630d2a4aba53498b8e6f40d713fe50d1",
      "placeholder": "​",
      "style": "IPY_MODEL_313d214425c541b6845f70638f8a5937",
      "value": " 24.9k/24.9k [00:00&lt;00:00, 18.5kB/s]"
     }
    },
    "8de92b21ea1b4dfc966d653e0ca7043b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9059a60d370c4825a0eb1b31c9f9cd0a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_db1af11912fa4774b718242b8861a18c",
      "max": 366,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2e0511f2f1844758a6f6d10af7233a72",
      "value": 366
     }
    },
    "94aa0a38801e4fa0b79150fbd7bcc6cc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "956dc45f3dde496d94f2d2520f62a241": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_dc1ad0316a634e0e81eecdd09d271136",
       "IPY_MODEL_5c531b5b3ce4406abde898297f4bc80b",
       "IPY_MODEL_8b30e8b778fb456bbf8af4d6931f415c"
      ],
      "layout": "IPY_MODEL_d1f7dc29912b40e7ac4a9081a4068836"
     }
    },
    "966ca413bd824a77b50a20784e8bd1b4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_044030ba2c5948d2ac6545aea773ef4a",
      "max": 199791,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a54e00389ce148829b4db5b7439ae879",
      "value": 199791
     }
    },
    "96acfb4ec404421b8523e39b6a4bc41a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9c573dc9c14248deb512162af5f9b0fa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a3384254326f49eb907bfa6d2032b1a8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a54e00389ce148829b4db5b7439ae879": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a70e5dadc1d547cc90da492ef9b72172": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a79295d7dc77434282636a0ef665a70b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a8b516bf196946a09a55732d937dfe2a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "addaf03f64e9421ab8debbba4f096d7e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "af5add21a56b4ef4945a90fed06d7c89": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bcaa24789c664ed181eb3af6c93a618e",
      "placeholder": "​",
      "style": "IPY_MODEL_e4015a34ae9744ceaa57d8b50a024d9a",
      "value": "Downloading readme: 100%"
     }
    },
    "b0b4e07185574c5eba5144a28c611971": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b2eeaecdd52343a4977954915f38bc9b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_726309e90e0247628487e1f4816e6fd9",
      "placeholder": "​",
      "style": "IPY_MODEL_e791aeda253f40d3bf28da3aaf0ec3f3",
      "value": " 68.1k/68.1k [00:00&lt;00:00, 93.5kB/s]"
     }
    },
    "b65097925371428eb2b58eec672f9b81": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "b68b7b7b072e457980e834113ff0a670": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b7e131bcb6454ccbb5de0813a734586a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b9f43527572c41849f66e100e4d2df83": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a70e5dadc1d547cc90da492ef9b72172",
      "max": 7544,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_3ff0a924499f414ca7fd03a902b51102",
      "value": 7544
     }
    },
    "bcaa24789c664ed181eb3af6c93a618e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c0822d9ca57b4b23ba552b93f139d96e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_23207dda3c7c45ef942fb26b6d3e8f4a",
      "max": 31241,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_0a4bcc907dbc48f3af39fac4c0652bd4",
      "value": 31241
     }
    },
    "c55cb9a98afe49d59838a3e2d88cd774": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c56009b340984f2296ad2959e8d72f25": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c6fa8ced2f844e28a6e5cf4b31e362b1",
      "max": 68082,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b65097925371428eb2b58eec672f9b81",
      "value": 68082
     }
    },
    "c6fa8ced2f844e28a6e5cf4b31e362b1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d0cecf202f814209abe965e2ec13c49e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d1f7dc29912b40e7ac4a9081a4068836": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d2e3caaeded749fc807de397f79e98e7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_09cc068788e64c1fbc1edd0992f43f42",
      "placeholder": "​",
      "style": "IPY_MODEL_46e05d2996a84b5e8f456adbb6453350",
      "value": "Generating validation split:  95%"
     }
    },
    "d7be4847987e4d6996c56eb42feb78b9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d8a6361f19894be1b730137b0801e4e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_764abccaae30410ba68405f44875c8d6",
      "placeholder": "​",
      "style": "IPY_MODEL_89dd9177f7cd4edc87e43d5e23c5f587",
      "value": "100%"
     }
    },
    "db1af11912fa4774b718242b8861a18c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dbb568c5b348421a9e2f9f4e07cd5f01": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_addaf03f64e9421ab8debbba4f096d7e",
      "placeholder": "​",
      "style": "IPY_MODEL_94aa0a38801e4fa0b79150fbd7bcc6cc",
      "value": "Downloading data: 100%"
     }
    },
    "dc1ad0316a634e0e81eecdd09d271136": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f8af035c2033430ca41cb3c81719ddc8",
      "placeholder": "​",
      "style": "IPY_MODEL_682be41e57c341189bdc3e7a6b7246d3",
      "value": "Downloading builder script: 100%"
     }
    },
    "df02d3127e634862b1029787be8908b8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e3641728ace74d8ab4742007d1725ee8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": "hidden",
      "width": null
     }
    },
    "e4015a34ae9744ceaa57d8b50a024d9a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e791aeda253f40d3bf28da3aaf0ec3f3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e81a3f25949247eeb696ccd3b0df3da0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_435416a295674774a15aa700ff7ce007",
      "placeholder": "​",
      "style": "IPY_MODEL_d7be4847987e4d6996c56eb42feb78b9",
      "value": " 200k/200k [00:00&lt;00:00, 269kB/s]"
     }
    },
    "ea1af7f963e848d19ea363331027109b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d0cecf202f814209abe965e2ec13c49e",
      "placeholder": "​",
      "style": "IPY_MODEL_331752b5913245e380752532ab15f128",
      "value": " 3/3 [00:00&lt;00:00, 36.01it/s]"
     }
    },
    "eaa70494878f4eba927868ecbbd034b4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b0b4e07185574c5eba5144a28c611971",
      "placeholder": "​",
      "style": "IPY_MODEL_01a6160718434f7bb73bb4a094ee8b0f",
      "value": "Downloading extra modules: 100%"
     }
    },
    "ed32d5154c8e45ff91608c8dca664d8e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_d2e3caaeded749fc807de397f79e98e7",
       "IPY_MODEL_f9d799cf26574229ad1d77373cb511c5",
       "IPY_MODEL_17a314c74019462e9ba2e923b9235bed"
      ],
      "layout": "IPY_MODEL_2435607c37f2439aa1f0d545009b375e"
     }
    },
    "ed58c5b583f344cdbfea76562259190e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_672ee2ad96ac42fa90645a4049c7dd16",
      "placeholder": "​",
      "style": "IPY_MODEL_96acfb4ec404421b8523e39b6a4bc41a",
      "value": "Downloading metadata: 100%"
     }
    },
    "f0019a4884124eebb44571ca0a7b262e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0452728577c4433d9d15eca97045d1f4",
      "placeholder": "​",
      "style": "IPY_MODEL_53938941490b4afea31c63aad4bdce0a",
      "value": " 7.54k/7.54k [00:00&lt;00:00, 8.92kB/s]"
     }
    },
    "f8af035c2033430ca41cb3c81719ddc8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f9d799cf26574229ad1d77373cb511c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7cc6a3332eaf4e9bb012097ca03c6ef6",
      "max": 366,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7bb9e900006840a7a892a1a48d676560",
      "value": 366
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
